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DAM /CAE  (Overall  Composite  Voice  Quality  Score)  for  the 
DOD  LPC-10  Vocoder  in  the  E4B  Battlestaff  Compartment 
Acoustic  Environment  (Resident  Microphone) 
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DAM/CAE  (Overall  Composite  Voice  Quality  Score)  for  the 
DOD  LPC-10  Vocoder  in  the  E4B  NCA  Compartment  Acoustic 
Environment  (Resident  Microphone) 
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DAM/CAE  (Overall  Composite  Voice  Quality  Score)  for  the 
DOD  LPC-10  Vocoder  in  the  E4B  Briefing  Room  Acoustic 
Environment  (Resident  Microphone) 
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A7. 


DAM/CSA  (Signal  Quality  Scores)  for  the  DOD  LPC-10 
Narrowband  Vocoder  in  the  E4B  Battlestaff  Compartment 
Acoustic  Environment  (Resident  Microphone) 
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A8. 

A9. 


DAM/CBA  (Background  Quality  Scores)  for  the  DOD  LPC-10 
Narrowband  Vocoder  in  the  E4B  Battlestaff  Compartment 
Acoustic  Environment  (Resident  Microphone) 
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DAM/CSA  (Signal  Quality  Scores)  for  the  DOD  LPC-10 
Narrowband  Vocoder  in  the  E4B  NCA  Compartment 
Acoustic  Environment  (Resident  Microphone) 
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A10.  DAM/CBA  (Background  Quality  Scores)  for  the  DOD  LPC-10 

Narrowband  Vocoder  in  the  E4B  NCA  Compartment  Acoustic- 
Environment  (Resident  Microphone) 

All.  DAM/CSA  (Signal  Quality  Scores)  for  the  DOD  LPC-10 

Narrowband  Vocoder  in  the  E4B  Briefing  Room  Acoustic 
Encironment  (Resident  Microphone) 

A12.  DAM/CBA  (Background  Quality  Scores)  for  the  DOD  LPC-10 
Narrowband  Vocoder  in  the  E4B  Briefing  Room  Acoustic 
Environment  (Resident  Microphone) 

A 13.  DRT  Intelligibility  Scores  for  the  DOD  LPC-10  Narrowband 

Vocoder  in  a  Quiet  Acoustic  Environment  (Dynamic  Microphone) 

A14.  DAM/CAE  (Overall  Composite  Voice  Quality  Scores)  for  the 
DOD  LPC-10  Narrowband  Vocoder  in  a  Quiet  Acoustic 
Environment  (Dynamic  Microphone) 

A  15.  DAM/CSA  (Signal  Quality  Scores)  for  the  DOD  LPC-10 
Narrowband  Vocoder  in  a  Quiet  Acoustic  Environment 
(Dynamic  Microphone) 

A16.  DAM/CBA  (Background  Quality  Scores)  for  the  DOD  LPC-10 
Narrowband  Vocoder  in  a  Quiet  Acoustic  Environment 
(Dynamic  Microphone) 

A17.  DRT  Intelligibility  Scores  for  the  CVSD-16  Processor  in  a 
Quiet  Acoustic  Environment  (Dynamic  Microphone) 

A18.  DAM/CAE  (Overall  Composite  Voice  Quality)  Scores  for  the 
CVSD-16  Processor  in  a  Quiet  Acoustic  Environment 
(Dynamic  Microphone) 

A19.  DAM/CSA  (Signal  Quality)  Scores  for  the  CVSD-16  Processor 
in  a  Quiet  Acoustic  Environment  (Dynamic  Microphone) 

A20.  DAM/CBA  (Background  Quality)  Scores  for  the  CVSD-16 
Processor  in  a  Quiet  Acoustic  Environment  (Dynamic 
Microphone) 

A21.  DRT  Intelligibility  Scores  for  the  CVSD-32  Processor  in 
a  Quiet  Acoustic  Environment  (Dynamic  Microphone' 

A22.  DAM/CAE  (Overall  Composite  Voice  Quality)  Scores  for  the 
CVSP-32  Processor  in  a  Quiet  Acoustic  Environment 
(Dynamic  Microphone) 

Bl.  List  of  Master  Tapes:  E4B  Battlestaff  Compartment  Acoustic 
Environment  (Roanwell  Model  240100001  Noise  C  ancelling 
Microphone 

B2.  List  of  Master  Tapes:  E4B  NCA  Compartment  Acoustic 

Environment  (Roanwell  Model  240)00001  Noise  Cancelling 
Microphone) 

B3.  List  of  Master  Tapes:  E4B  Briefing  Room  Acoustic  Environment 
(Roanwell  Model  240100001  Noise  Cancelling  Microphone) 

B4.  List  of  Master  Tapes;  EC  -135  Battlestaff  Compartment  A  nous! 
Environment  (Roanwell  Model  60150  Microphone) 

B5.  List  of  Master  Tapes:  EC-135  Radio  Compartment  Acoustic 
Envir' >nrnent  (Roanv.ell  Model  601  50  Microphone) 


Narrowband  (LPC-10)  Vocoder  Performance 
Under  Combined  Effects  of  Random  Bit 
Errors  and  Jet  Aircraft  Cabin  Noise 


I.  INTRODICTION 

The  performance  of  digital  voice  communications  processors  (vocoders,  etc) 
over  degraded  communications  channels,  such  as  channels  where  data  errors  are 
incurred  because  of  interference,  power  or  bandwidth  limitations,  or  other  causes, 
is  of  prime  importance  in  designing  communications  systems.  Therefore,  tests  to 
determine  the  suitability  of  a  voice  processor  for  tactical  or  strategic  communica¬ 
tions  usually  include  speech  intelligibility  and  voice  quality  evaluations  under  im¬ 
pairments  caused  by  bit  errors  in  the  data  path. 

1.1  Linear  Regression  Modeling 

It  has  been  determined  that  speech  intelligibility  scores  and  voice  quality 

scores  from  tests  of  vocoders  and  wideband  voice  processors  under  random  bit  er- 

1  2 

ror  conditions  tend  to  follow  a  linear  relationship  with  bit  error  rate.  '  The 
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microphones  and  the  particular  male  and  female  speakers  used  for  testing  are  fac¬ 
tors  that  can  impart  an  overall  bias  to  scores.  However,  assuming  that  tests  have 
been  conducted  at  several  bit-error  rates  with  these  factors  held  constant,  volumi¬ 
nous  test  data  can  be  reduced  to  simple  linear  expressions  of  the  form 

S'  =  A  +  Bx 

where  S  provides  an  estimate  of  average  speech  intelligibility  score  ;or  voice  qual¬ 
ity  score);  x  -  bit  error  rate  in  percent;  and  the  regression  coefficients  A  and  B 
express  the  origin  and  slope,  respectively,  of  a  linear  function  estimating  perfor¬ 
mance  vs  bit  error  rate  that  is  fitter!  to  the  data  values.  The  coefficient  A  ex¬ 
presses  the  expected  score  with  no  bit  errors,  the  origin  of  the  regression  line. 

The  coefficient  B  expresses  the  slope  of  the  line,  the  amount  tha’  ‘  o  score  will 
drop  with  each  percent  bit  error  rate.  For  example,  confidence  limits  calculated 
from  the  data  values  can  establish  the  L'5  percent  probability  bounds  of  the  pre¬ 
dicted  average  score  at  any  error  rate,  and  the  confidence  bounds  for  the  data  pop¬ 
ulation,  that  is,  the  collection  of  intelligibility  scores  or  quality  scores  from  which 
the  regression  model  was  calculated.^ 

1.2  Combined  Hit-Urror  and  Acoustic  Kin ironmcnt  HffccN 

In  earlier  studies,  extensive  tests  of  vocoders  and  wideband  speech  processors 
[continuous  variable-slope  delta  modulation,  or  CVSD,  at  lfi  kilobits  per  second 
(K  bps)  and  32  K  bps  data  rates]  were  conducted  under  random  and  block  errors 
with  "ideal"  input  speech  signals,  that  is,  speech  from  a  quiet  environment  and 
through  a  high  quality  microphone.  The  studv  reported  here  evaluated  the  combine* 
effects  of  random  errors  imposed  on  the  vocoder  data  and  of  noise  combined  with 
the  input  speech,  noise  representing  the  acoustic  environments  found  in  the  cabins 
of  jet  aircraft  during  flight.  The  voice  processor  tested  was  the  Department  of 
Defense  (DOD)  standard  LPC-10  narrowband  vocoder  algorithm. Acoustic  noise 
environments  were  those  measured  in  K4B  and  FC  - 1 35  aircraft  during  flight.  Mi¬ 
crophones  used  for  the  tests  were  the  resident  micron:  ones,  the  handset  instru¬ 
ments  installed  in  these  aircraft.  Performance  data  from  earlier  evaluations  of 
voice  processor  performance  under  bit  error  impairments  with  speakers  in  a  quiet 
enviromnent  are  also  presented  in  this  report,  for  comparison. 

3.  Snedecor,  G.W.,  and  Cochran,  W.G.  i  1967)  Statistical  Methods.  Iowa  State 

University  Press,  Ames,  Iowa. 

4.  T remain,  T.  *1982)  The  government  standard  linear  preuivave  coning  algo¬ 

rithm:  1. PC-10,  Speech  Tech.  I(V>.  2h 


1.3  Background 


This  study  was  conducted  by  the  COMSEC  Engineering  Office  HIADC/EHV'. 
Electromagnetic  Sciences  Division,  RADC,  Hanscom  AFB,  Mass...  under  sponsor¬ 
ship  of  the  AF  Electronic  Systems  Division  (ESD/SCS).  The  work  involved  a  team 
effort,  with  contractor  participation  provided  by  Ketron,  Inc.,  Wayne,  Pa.,  and 
by  Dynast  at,  Inc.,  Austin,  Tex.  Mr.  Charles  Teacher,  Ketron's  principal  inves¬ 
tigator,  made  significant  contributions  by  planning  and  participating  in  the  opera¬ 
tional  flights.  He  had  the  lead  role  in  taking  measurements  aboard  the  aircraft, 
collecting  data,  and  making  live  speech  recordings.  Mr.  Teacher  also  analyzed 
the  acoustic  environments  and  microphone  characteristics. 

Dynastat  furnished  the  standard  speakers  used  for  live  speech  recordings 
aboard  the  aircraft  and  prepared  master  tapes  of  simulations  of  the  acoustic  envi¬ 
ronments.  Here,  "standard"  speakers  refers  to  individuals  whose  voices  are  re¬ 
corded  in  an  extensive  DOD  tape  library  of  speech  materials  for  intelligibility  and 
voice  quality  testing,  and  on  whom  extensive  data  has  been  collected  from  evalua¬ 
tions  of  the  performance  of  DOD  and  industry  voice  processors  under  many  test 
conditions.  Dynastat  conducted  tests  with  listener  crews  to  determine  speech  in¬ 
telligibility  and  voice  quality,  using  government -furnished  recordings  prepared  at 
the  COMSEC  Engineering  Office,  RADC/EEV  while  testing  the  DOD  standard  vocoder. 

RADC/EEV  provided  overall  management  and  direction  of  the  effort  as  well  as 
the  data  analyses  and  interpretations  reported  here. 

2.  TASKS 

Accomplishment  of  this  work  required  the  following  sequence  of  tasks: 

2.1  Characterization  of  the  Acoustic  Km  ironmcnts  Aboard  the  Aircraft 

HQ  SAC  allowed  participation  in  flights  from  Offut  AFB,  Nebr.,  in  June  and 
July  1982.  During  those  flights  on  E4B  and  EC-135  aircraft,  the  following  tasks 
were  accomplished: 

(1)  Sound  pressure  levels  were  me  isured  at  several  positions  in  each  aircraft. 

(2)  Live  recordings  of  speech  intelligibility  test  materials  and  voice  quality 
test  materials  were  prepared,  using  as  speakers  Mr.  Teacher  from 
Ketron  and  speakers  provided  by  Dynastat. 

(3)  Acoustic  background  noise  in  the  cabins  of  the  aircraft  was  recorded  with 


5.  Teacher,  C.F.  (1983)  Secure  Voice  Evaluations:  Characterization  of  Aircraft 
Noise  and  Audio  Systems  in  the  EC-135  and  E4B  Aircraft,  Report  KET- 
2709-2,  Ketron,  Inc.,  Wayne,  Pa. 
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a  portable  stereo  tape  recorder  for  subsequent  uses  in  simulating  the 
acoustic  environments. 


2.2  Preparation  of  Master  Tapes  for  Speech  Tests  and  Evaluations 


The  number  of  live  speech  test  recordings  aboard  the  aircraft  was  extremely 
limited.  Therefore,  Dynastat  expanded  speech  intelligibility  and  voice  quality  test 
materials  representative  of  the  E4B  Battlestaff  Compartment,  Briefing  Room,  and 
NCA  Compartment,  and  of  the  EC-135  Battlestaff  Compartment  and  Radio  Compart¬ 
ment  acoustic  environments  by  preparing  simulations.  Acoustic  environments 
were  reproduced  in  a  sound  room  by  reproducing  the  in-flight  recordings  of  cabin 
noise,  with  the  sound  pressure  level  of  the  reproduced  cabin  noise  adjusted  to 
match  the  levels  measured  aboard  the  aircraft.  The  stereo  noise  tapes  were  re¬ 
produced  with  two  loudspeakers  placed  in  corners  of  the  room.  A  measurement 
and  calibration  procedure  verified  that  the  reproduced  noise  field  was  relatively 
uniform  in  the  vicinity  of  the  speaker  subjects  and  their  microphones,  and  that  the 
sound  pressure  levels  of  the  simulations  reproduced  those  measured  aboard  the 
aircraft.  Normal  sidetone  signals  were  furnished  for  the  speaker  subjects  during 
the  recordings.  Master  tapes  prepared  in  these  simulations  are  listed  in  Appen¬ 
dix  B. 


2.2.1  MICROPHONES 


Microphones  were  mounted  in  handsets  and  clamped  in  supports  keeping  them 
at  the  optimum  placement  from  the  speaker's  lips. 

For  recordings  in  the  E4B  environment,  a  Roanwell  Model  240100001  pressure- 
gradient  (noise  cancelling)  microphone  mounted  in  a  handset  was  used.  This  mi¬ 
crophone  is  used  for  command  and  control  functions  on  the  E4B;  it  is  referred  to 
in  this  report  as  the  "E4B  resident  microphone.  " 

For  the  EC-135  acoustic  environment  test,  a  Roanwell  Model  60150  pressure 
microphone  mounted  in  a  handset,  the  command  and  control  microphone  used  on 
that  aircraft,  was  used.  The  Model  60150  is  not  a  noise  cancelling  microphone, 
and,  while  test  results  indicated  that  speech  intelligibility  of  the  vocoder  was  simi¬ 
lar  in  the  two  aircraft,  the  voice  quality  ratings  obtained  in  the  EC-135  acoustic 
environment  were  poorer  than  those  in  the  E4B,  probably  because  the  speech-to- 
noise  ratio  was  poorer. 


2.2.2  SPEAKER  SUBJECTS 


Six  male  and  three  female  speakers  were  speaker  subjects  in  the  simulations 
of  the  acoustic  environments;  of  these,  three  of  the  male  speakers  were  common  to 
the  live  recordings  aboard  the  aircraft.  Four  "scramblings"  (randomizations)  of 


the  Diagnostic  Rhyme  Test  (DRT)“*  intelligibility  word  Msts  were  prepared  for 
each  speaker /test -condition  combination.  The  design  of  the  DRT  intelligibility  test 
makes  it  unlikely  that  subjects  will  memorize  word  lists  with  repeated  presenta¬ 
tions;  for  these  studies,  the  many  equivalent  versions  of  word  lists  guaranteed  that 
memorization  would  not  occur. 

2.3  \  orodrr  Tests 

Vocoder  tests  were  carried  out  at  RADC/EEV's  speech  test  and  evaluation  fa¬ 
cility  at  Hanscom  AFB,  Mass.  The  facility  has  been  used  for  voice  processor 
tests  for  the  DOD  Digital  Voice  Processor  Consortium  over  the  past  3  years.  It  is 
equipped  with  professional  tape  recorders,  random  bit -error  generators,  a  hard¬ 
ware  vocoder  that  incorporates  the  DOD  standard  LPC-10  narrowband  vocoder  al¬ 
gorithm,  wideband  16 K  bps  and  32 K  bps  CVSD  voice  processors,  and  the  DOD  li¬ 
brary  of  master  tape  recordings  prepared  specifically  for  testing  speech  proces¬ 
sors  for  intelligibility  and  quality.  Bit  error  tests  were  conducted  with  random 
errors  at  0-,  1/2-,  1-,  2-,  and  5-percent  error  rates,  with  replication  of  the  test 
at  each  error  rate  using  different  scramblings  of  the  DRT  word  lists. 

2.  I  Listener  Tests 

For  listener  tests,  tape  recordings  of  the  vocoder  speech  were  prepared  at 
RADC/EEV  and  forwarded  to  Dynastat  for  presentation  to  listener  crews  of  eight  or 
more  subjects.  The  speech  signals  were  presented  diotically  (to  both  ears)  over 
headphones,  with  subjects  assembled  in  an  acoustically  treated  room.  Listener 
tests  were  "blind":  neither  the  listeners  nor  test  administrators  knew  the  identity 
of  processors  or  test  conditions  under  evaluation.  Test  materials  were  identified 
only  by  the  assigned  serial  numbers,  a  procedure  routinely  used  to  guarantee  the 
integrity  of  test  results. 

2.5  Analysis  of  Performance  Data 

Diagnostic  scores  and  overall  scores  were  determined  from  listener  responses 
by  Dynastat.  Listings  of  diagnostic  scores  and  voice  quality  ratings  with  standard 
errors  were  supplied  to  RADC. 

6.  Voiers,  W.D. ;  Cohen,  M.E. ;  and  Mickunas,  J.  (1965)  Evaluation  of  Speech 

Processing  Devices:  1.  Intelligibility.  Quality  and  Speaker  Recognizability. 
AFCRL-65-826,  Final  Report,  Contract  AF19(628)-4195,  AD  627320. 

7.  Voiers,  W.D.  (1977)  Diagnostic  evaluation  of  speech  intelligibility,  in  Speech 

Intelligibility  and  Speaker  Recognition.  2,  M.E.  Hawley,  Ed.,  Benchmark 
Papers  in  Acoustics,  Dowden,  Hutchinson  and  Ross,  Stroudsburg,  Pa. 

8.  Voiers,  W.D.  (1983)  Evaluating  processed  speech  using  the  Diagnostic  Rhyme 

Test,  Speech  Tech.  l(No.  3). 


The  performance  data  were  analyzed  at  RADt  for  the  following  assessments • 

(1)  Similarities  of  acoustic  environments  m  tne  aircraft  compartments; 

(2)  Comparisons  of  live  and  simulated  environments; 

(3)  Computation  of  regression  models  expressing  the  relationships  bvtwctr 
performance  and  error  rate;  and 

(4)  Comparisons  of  acoustic  environments  in  tl."  aircraft  with  a  quiet  envi¬ 
ronment. 

3.  \l*l’RO\<  II 

As  vocoder  equipments  nave  become  more  sophisticated  and  cotrnlex,  testing 
procedures  have  become  refined,  and  informal  listening  judgments  l.-\  experts 
have  been  replaced  by  standardized  ntuHi  pie  -speaker  diagnostic  test-  presented  >c 
listener  crews  of  eight  or  more  subjects.  After  frustrating  experiences  witi  older 
channel -vocoders,  we  appreciated  that  gooa  speech  intelligibility  was  p  nee-,  ssar 
but  not  sufficient  attribute  for  measuring  user  acceptance,  speecn  quality  tests 
were  developed  to  supplement  intelligibility  tests,  *  and  these,  in  turn,  have  been 
refined  to  provide  diagnostic  information  on  speech  qualitv.  " 

3.1  Tin*  Diagnostic  Rliinic  Tt>l  (l)RT) 

Over  the  years,  vocoder  intelligibility  has  been  evaluated  with  phonetically 
balanced  (PE)  word  Lists,  the  Fairbanks  Rhyme  Test,  the  Modified  Rhym»  Test 
(MRT),  ^  the  C'onsonant  Rhyme  Test  (CRT),  *  ■  and  the  Diagnostic  Rhvme  i  est 
(DKT).  y  Of  these,  the  most  widely  used  in  the  past  decade  has  been  the  DRT 
of  Voiers,  Cohen,  and  Mickunas,  ®  which,  has  become  the  preferred  intelligibility 
test  method  of  the  DOD  Digital  Voice  Processor  Consortium.  Numerous  tests  o' 
voice  processors  have  shown  that  any  processor  that  obtains  a  high  intelligibility 
score  on  multiple -speaker  tests  by  this  method  will  be  capable  of  providing  highlv 

♦Speech  quality  tests  supplement,  rather  than  replace,  intelligibility  tests  because 
high  " quality' ’  scores  do  not  necessarily  imply  higr.  intelligibility.  I'  or  exar  nb  , 
soeech  low-pass  filtered  at  2  kHz  results  in  high  quality  scores  ever,  though  in¬ 
telligibility  is  significantly  impaired. 
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intelligible  speech  communications  for  ordinary  conversational  speech,  and  will 
provide  even  higher  margins  of  performance  when  messages  are  highlv  stereo¬ 
typed,  a  condition  typical  of  military  communications,  where  context  and  redun¬ 
dancy  improve  intelligibility. 

3.2  Categoric*  of  DRT  Intelligibility  Scores 

The  DOD  Digital  Voice  Processor  Consortium  has  now  established  a  scale  of 
descriptive  categories  for  DRT  intelligibility  scores.  The  categories  describe  rhe 
level  of  intelligibility  associated  with  different  ranges  of  intelligibility  scores  and 
are  illustrated  in  Table  1.  Scores  of  96  and  higher  are  categorized  as  "excellent.' 
Such  scores  are  obtained  with  high  quality  speech  under  quiet  conditions,  such  as 
the  DRT  master  tapes  used  in  evaluating  "ideal"  conditions.  At  the  other  extreme, 
scores  below  70  are  categorized  as  "unacceptable."  An  example  of  unaccept  ability 
is  vocoder  intelligibility  with  a  speaker  in  the  noise  environment  of  a  military  he¬ 
licopter.  Between  these  extremes  are  categories  "very  good"  for  scores  from  96 
to  91;  "good"  for  scores  from  91  to  87,  obtained  with  typical  commercial  telephony 
within  the  continental  United  States;  "moderate"  for  scores  from  87  to  83;  "fair 
for  the  range  from  83  to  79;  and  "poor"  for  scores  from  79  to  75,  Intelligibilii  , 
scores  between  75  and  70  are  categorized  as  "very  poor." 

These  values  represent  a  consensus  of  the  DOD  Digital  Voice  Processor  Con¬ 
sortium  Test  and  Evaluation  Committee.  Further  work  is  required  to  validate 
these  values  with  judgments  of  operational  personnel. 

3.3  twice  Quality  Tests:  The  Diagnostic  Acceptability  Measure  (DAM)  Test 

Each  vocoder  test  involved  paired  intelligibility  and  voice  quality  tests  using 
identical  speakers  and  conditions.  Speech  samples  for  evaluating  quality  used  con¬ 
nected  speech  rather  than  words  spoken  in  isolation.  In  the  Diagnostic  Acceptabi¬ 
lity  Measure  (DAM)  test  of  Voiers,  ®  the  processed  speech  samples  were  presented 
to  listener  crews  who  made  judgments  of  quality  attributes  of  the  speech  signal  and 
the  background  noise,  caused  by  the  combination  of  bit  errors  and  acoustic  noise 
environment,  that  accompanied  the  signal. 

Dynastat's  analyses  of  listener  responses  were  incorporated  into  data  summa¬ 
ries  that  included  scores  for  speech  signal  quality,  designated  Composite  Signal 
Acceptability  (DAM/CAS)  scores;  background  signal  quality,  designated  Composite 
Background  Acceptability  (DAM/CBA)  scores;  and  overall  composite  quality,  de¬ 
signated  Composite  Acceptability  Estimate  (DAM /CAE)  score,  a  weighted  combi¬ 
nation  of  the  CSA  and  CBA  scores.  (Overall  quality  can  be  estimated  from  the 
DAM/CSA  and  DAM/CBA  scores  by  expressing  the  scores  as  fractions  and  multi¬ 
plying  the  CSA  and  CBA  scores.  For  example,  with  a  CSA  score  of  80  and  a  CBA 


Table  1.  Categories  of  DRT  Intelligibility  Scores  With  Examples  Based  on 
Typical  DRT  Scores  for  Male  Speakers 


Category 

Examples 

Qualifiers  for  these  examples 

Excellent 

Unfiltered  speech 

Speech  low-pass  filtered 
at  4  kHz 

Speech  from  a  quiet 
environment:  no  significant 
distortions;  high-quality 
microphone 

Very  Good 

CVSD  at  32  K  bps 

CVSD  at  16  K  bps 

Error  rate  less  than  1%; 
speech  from  a  quiet 
environment 

Good 

Typical  commercial  telephony 
within  continental  USA 

Speech  from  a  quiet 
environment 

APC  Processor  at  9600  bps 

LPC-10  Vocoder  at  2400  bps, 
no  bit  errors 

Moderate 

LPC-10  Vocoder  with  bit  error 
protection,  at  2400  bps  with 

2%  random  bit  errors 

Speech  from  a  quiet 
environment 

Fair 

LPC-10  Vocoder  without  bit 
error  protection,  at  2400  bps 
with  2 %  random  bit  errors 

Speech  from  a  quiet 
environment 

Poor 

LPC-10  Vocoder  with  bit  error 
protection,  at  2400  bps  with 

5%  random  bit  errors 

Speech  from  a  quiet 
environment 

Very  Poor 

Experimental  800  bps  voice 
processor  with  no  bit  errors 

Speech  from  a  quiet 
environment 

Unacceptable 

LPC-10  Vocoder  at  2400  bps 

Speech  from  a  helicopter 
noise  environment 

DRT 

Score 


100 


96 


91 


87 


83 


78 


75 


70 


score  of  70,  multiplying  0.  80  by  0. 70  results  in  0.  56,  or  a  DAM/CAE  score  of 
approximately  56. ) 


3.1  Categories  of  DAM/CAE  Voice  Qualih  Scores 

Interpretation  of  voice  quality  scores  is  more  difficult  than  interpretation  of 
intelligibility  scores.  Quality  scores  do  not  cover  a  range  from  0  to  100,  but  typ¬ 
ically  occur  over  a  more  restricted  range  from  about  20  to  70.  A  DAM /CAE  score 


of  20  represents  extremely  distorted  and  noisy  speech  (a  score  below  30  generally 
implies  unacceptable  speech  quality).  A  DAM/CAE  score  of  70  represents  speech 
quality  approaching  "high  fidelity.  " 

Our  experience  in  interpreting  numerous  tests  has  resulted  in  assigning  the 
descriptive  categories  for  quality  scores  shown  in  Table  2.  Voice  quality  (DAM/- 
CAE)  scores  above  64  are  categorized  as  "excellent,  "  and  scores  from  64  to  58 
are  "very  good";  scores  in  the  64-58  range  have  been  obtained  for  a  32  K  bps  CVSD 
voice  processor  with  no  bit  errors,  and  input  speech  from  a  quiet  environment  and 
a  high  quality  microphone.  Scores  for  the  same  processor  drop  to  the  "moderate" 
range,  between  53  and  48,  when  the  speech  is  from  an  "office  noise"  environment. 
Quality  scores  in  the  range  from  48  to  42  are  categorized  as  "fair."  Examples 
are  the  scores  obtained  for  a  16  K  bps  CVSD  processor  at  5  percent  random  bit 


Table  2.  Categories  of  DAM/CAE  Voice  Quality  Scores  With  Examples  Based  on 
Typical  DAM /CAE  Scores  for  Male  Speakers 


1 - — - — - - 

DAM/CAE 

Score 

Category 

Examples 

Qualifiers  for  these  examples 

64 


58 


53 


48 


42 


36 


30 


Excellent  High  fidelity  speech 


From  a  quiet  environment 


Very  Good 

CVSD  at  32  K  bps,  with 
no  bit  errors 

Speech  from  a  quiet 
environment 

Good 

CVSD  at  16  K  bps,  with 
no  bit  errors 

Speech  from  a  quiet 
environment 

Moderate 

CVSD  at  16K  bps,  wtth 
no  bit  errors 

Speech  from  an  office  noise 
environment 

Fair 

CVSD  at  16K  bps,  with  5% 
random  bit  errors 

LPC-10  Vocoder  wtth  bit 
error  protection,  at  2400  bps 
with  1%  random  btt  errors 

Speech  from  a  quiet 
environment 

Poor 

LPC-10  Vocoder  with  btt 
error  protection,  at  2400  bps 
with  2%  random  bit  errors 

Speech  from  a  quiet 
environment 

Very  Poor 

Experimental  800  bps  voice 
processor,  with  no  bit  errors 

Speech  from  a  quiet 
environment 

Unacceptable 

LPC-10  Vocoder  at  2400  bps 

Speech  from  a  helicopter 
noise  environment 
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errors,  or  the  DOD  standard  LPC-10  narrowband  vocoder  with  1  percent  random 
bit  errors,  with  speech  from  a  quiet  acoustic  environment  in  both  examples.  Be¬ 
tween  42  and  36,  quality  scores  are  categorized  as  "poor."  One  example  is  the 
quality  (DAM/CAE)  score  obtained  for  the  DOD  standard  LPC-10  vocoder  at  a 
2  percent  random  bit  error  rate,  where  the  input  speech  is  from  a  quiet  environ¬ 
ment  and  through  a  high  quality  dynamic  microphone.  The  range  of  quality  scores 
from  36  to  30  is  categorized  as  "very  poor."  An  example  is  an  experimental  voice 
processor  operating  error-free  at  800  bps  with  a  high-quality  input  speech  signal. 
Voice  quality  scores  below  30  are  considered  "unacceptable";  scores  in  this  range 
have  been  obtained  from  tests  of  the  DOD  standard  LPC-10  vocoder  where  the  in¬ 
put  speech,  at  a  very  poor  signal -to-noise  ratio,  came  from  a  military  helicopter 
noise  environment. 


.‘{..I  l)is<'ii*Mon  of  Categories  of  Scores 

Descriptive  categories  of  intelligibility  and  voice  quality  scores  presented  in 
Tables  1  and  2  are  proposed  as  an  aid  in  interpreting  evaluation  data.  The  desig¬ 
nated  scales  are  tentative  and  may  undergo  revisions  after  further  research  and 
experience.  For  example,  separate  scales  for  categories  may  be  appropriate  for 
distinguishing  between  levels  of  performance  associated  with  tactical  communica¬ 
tions  and  those  associated  with  strategic  and  executive  communications.  Speech 
quality  levels  rated  poor  for  executive  communications  may  be  rated  good  by  com¬ 
municators  in  the  tactical  world.  Additional  research  is  required  to  clarify  this 
question. 

L  CHARACTERIZATIONS  OF  MRCRAFT  NOISE  ENVIRONMENTS 

Teacher'*’  has  reported  the  results  of  measuring  the  background  noise  levels 
aboard  the  E4B  and  EC-135  aircraft;  the  measured  sound  pressure  levels  are  sum¬ 
marized  in  Table  3.  The  background  noise  levels  measured  in  the  E4B  Battlestaff 


Table  3.  Sound  Pressure  Levels  Measured 
Aboard  E4B  and  EC-135  Aircraft 

E4B  Battlestaff  Compartment  88  dbc 

E4B  NCA  Compartment  78  dbc 

E4B  Briefing  Room  84  dbc 


EC-135  Battlestaff  Compartment  89  dbc 

EC-135  Radio  Compartment  88.  5  dbc 


Compartment  and  Briefing  Room  closely  match  the  noise  levels  measured  in  an 
operational  flight  of  an  E3A  aircraft.  Simulations  of  the  acoustic  environments 
prepared  by  Dynastat  are  described  in  Section  2.2. 


5.  VOCODER  TESTS 
5. 1  Vocoder  Test  Setup 

The  setup  for  vocoder  testing  is  shown  in  Figure  1.  The  live  speech  recorded 
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Figure  1.  Block  Diagram  of  Vocoder  Test  Setup 


aboard  the  aircraft  and  master  tapes  prepared  by  simulating  the  acoustic  environ¬ 
ments  were  used  as  speech  input  materials  to  the  vocoder.  A  random  bit -error 
generator  created  errors  in  the  vocoder  data  path  at  the  specified  error  rates.  Out¬ 
put  speech  from  the  receiving  vocoder  was  recorded  for  subsequent  evaluations  by 
listener  crews.  With  every  bit-error  condition,  the  test  was  repeated  a  second 
time  with  different  randomizations  of  the  DRT  word  lists.  Table  4  lists  the  voco¬ 
der  test  conditions  and  master  tapes  used  for  each  condition. 

5.2  Live  vs  Simulated  Tests 

It  has  been  argued  that  testing  procedures  using  live  speech  input  are  superior 
to  those  using  speech  recordings.  The  distinction  is  not  clear-cut  because  several 
levels  of  "liveness"  must  be  considered.  At  one  extreme  of  test  authenticity,  the 
vocoder  could  be  taken  aboard  the  aircraft  with  teams  of  speaker  subjects  and  lis- 
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Table  4.  Schedule  of  Master  Tapes  Processed  for  Evaluating  the  DOD  I.PC-10 
Vocoder  Performance  in  the  E4B  Acoustic  Environment 


Zero  BER 

1/2%  BER 

1%  BER 

2%  BER 

5%  BER 

Battlestaff 

#9540 

#9546 

#9555 

#9563 

#9565 

Compartment 

K 1 0-1.  1 -A 

K10-1.3-A 

K 10-1. 1-A 

K10-1.2-A 

K10-1.  1-A 

Speakers: 

K10-1.2-B 

K10-1.4-B 

K10-1.3-B 

K 10-1 . 1-B 

K10-1.4-B 

6  Males 

K10-1.3-C 

K10-1.  1-C 

K10-1.3-C 

K10-1.4-C 

K  10-1.  1-C 

3  Females 

#9547 

#9541 

#9554 

#9567 

#95G2 

K 10-1. 4 -A 

K10-1.2-A 

K 10-1. 4 -A 

K 10-1. 4- A 

K10-1.  3-A 

K 10-1.  1-B 

K10-1.3-B 

K10-1.2-B 

K10-1.2-B 

K10-1.4-B 

K10-1.2-C 

K10-1.4-C 

K 10-1.  1-C 

K10-1.4-C 

K10-1.2-C 

NCA 

#9556 

#9548 

#9542 

#9553 

#9566 

Compartment 

K 10-2. 3 -A 

K10-2.3-A 

K10-2.  1-A 

K 10-2. 4 -A 

K  10-2.  3 -A 

Speakers: 

K10-2.3-B 

K10-2.4-B 

K 10-2. 2 -B 

K10-2.2-B 

K  1 0-2.  1-B 

6  Males 

K10-2.2-C 

K10-2.  1-C 

K10-2.3-C 

K  10-2.  1-C 

K10-2.4-C 

3  Females 

#9564 

#9557 

#9549 

#9543 

#9552 

K 10-2. 2 -A 

K10-2.  1-A 

K10-2.4-A 

K10-2.2-A 

I<  10-2.  2 -A 

K10-2.2-B 

K10-2.4-B 

K10-2.  1-B 

K 10-2.  3-B 

K 1 0-2.  1-B 

KlO-2. 1-C 

K10-2.4-C 

K10-2.2-C 

K10-2.4-C 

K10-2.3-C 

Briefing 

#9560 

#9568 

#9558 

#9550 

#9544 

Boom 

K10-3.4-A 

K10-3.3-A 

K 10-3. 3 -A 

K10-3.3-A 

K  1 0-3.  1-A 

Speakers: 

K10-3.3-B 

K10-3.3-B 

K10-3.2-B 

K10-3.2-B 

K  1 0-3.  2-B 

6  Males 

#9569 

#9559 

#9561 

#9545 

#9551 

K10-3.  1-A 

K10-3.2-A 

K 10-3. 1-A 

K10-3.2-A 

K10-3.4-A 

K10-3.4-B 

K 10-3. 1-B 

K10-3.4-B 

K 10-3. 3-B 

K 10-3. 1-B 

tener  subjects,  and  tests  could  be  run  during  a  flight.  However,  not  only  would 
this  procedure  be  very  costly,  but  also,  inevitably,  there  would  be  many  uncon¬ 
trolled  variables,  including  variations  in  speaker  effort,  microphone  placement, 
correctness  of  pronunciation  of  test  words,  voice  pitch,  fluctuations  in  the  acoustic 
environment,  etc.  The  result  would  be  inflated  standard  errors  and  the  impaired 
ability  to  make  close  comparisons  of  test  conditions  or  processor  hardware. 

At  a  reduced  level  of  authenticity  is  the  live  testing  procedure  used  here:  the 
speaker  subjects  were  recorded  aboard  the  aircraft  in  the  actual  acoustic  environ¬ 
ment,  and  their  tape  recordings  were  then  used  to  test  the  vocoder.  This  procedure 
is  much  less  costly  than  the  first,  but  is  still  subject  to  uncontrolled  variations. 

At  a  third  level,  speaker  subjects  and  listener  subjects  are  placed  in  simu¬ 
lated  acoustic  environments,  and  the  tests  are  conducted  live  in  a  sound  chamber. 
With  this  procedure,  greater  control  can  be  maintained  than  with  the  first  two. 

A  fourth  procedure,  using  recorded  speech  materials  prepared  under  closelv 
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controlled  conditions  and  validated  with  testing  and  comparisons  of  results,  makes 
it  possible  to  test  and  retest  processors  at  different  times  and  places  without  com¬ 
promising  the  ability  to  make  close  comparisons  of  results.  In  addition,  testing  is 
less  costly  because  the  formidable  logistics  problems  of  having  speaker  subjects 
and  listener  subjects,  vocoder  and  test  hardware,  sound  rooms,  and  test  person¬ 
nel  available  simultaneously  are  avoided. 

6.  INTELLIGIBILITY  OF  THE  DOD  LPC-IO  VOCODER 
IN  THE  E  IB  ACOrSTIC  ENV  IRONMENT 

Intelligibility  scores  obtained  for  male  and  female  speakers  from  testing  the 
DOD  standard  LPC-10  narrowband  vocoder  in  the  acoustic  environments  of  the 
E4B  Battlestaff  Compartment,  NCA  Compartment,  and  Briefing  Room  are  pre¬ 
sented  in  Tables  Al,  A2,  and  A3.  Intelligibility  scores  obtained  with  each  acous¬ 
tic  environment  were  analyzed,  and  linear  regression  models  for  the  relationship 
between  intelligibility  and  bit  error  rate  were  calculated.  The  individual  speaker's 
intelligibility  scores  listed  in  these  tables  are  presented  as  scatter  diagrams  in 
relation  to  the  computed  regression  line  and  95  percent  confidence  bounds  for  the 
data  populations  in  Figures  2-6. 


RANDOM  BIT  ERROR  RATE  (PERCENT) 


Figure  2.  Male  Speaker's  Intelligibility  Scores  vs  Bit  Error  Rate,  in 
Relation  to  Linear  Regression  Modeling,  for  the  DOD  LPC-10  Vocoder 
in  the  E4B  Battlestaff  Compartment  Acoustic  Environment 
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RANDOM  BIT  ERROR  RATE  (PERCENT) 

Figure  3.  Female  Speaker's  Intelligibility  Scores  vs  Bit  Error  Rate, 
in  Relation  to  Linear  Regression  Modeling,  for  the  DOD  LPC-10 
Vocoder  in  the  E4B  Battlestaff  Compartment  Acoustic  Environment 
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RANDOM  BIT  ERROR  RATE  (PERCENT) 


The  acoustic  environments  were  found  to  be  similar  in  their  effects  on  varia¬ 
tion  of  intelligibility  with  bit  error  rate.  Regression  lines  determined  for  scores 
from  the  three  environments  are  compared  in  Figure  7.  The  comparison  indicates 
that  the  lower  background  noise  level  of  the  E4B  NCA  Compartment  may  have  been 
involved  in  producing  a  small  advantage  in  intelligibility  at  the  higher  error  rates 
for  speech  from  that  environment.  However,  the  origins  of  the  regression  lines 
are  nearly  identical,  and  the  advantage  is  only  apparent  at  error  rates  where  in¬ 
telligibility  is  well  below  70,  a  threshold  value  for  minimum  acceptable  intelligi¬ 
bility.  Consequently,  the  different  between  regression  lines  determined  for  the 
NCA  Compartment  and  the  other  two  compartments  has  no  practical  significance. 


RANDOM  BIT  ERROR  RATE  (PERCENT) 


Figure  7.  Comparison  of  Expected  Variation  in  Intelligibility  With  Bit  Error  Rate 
for  the  DOD  LPC-10  Vocoder  Tested  in  the  Acoustic  Environments  of  the  Three 
E4B  Compartments 


6.1  Composite  Result  for  E1B  Aeou-lir  Environment:  Intelligibility  vs  Bit  Error  Rale 
Pooling  intelligibility  scores  for  the  three  environments  and  calculating  a  re- 


gression  model  for  the  composite  data  from  male  and  female  speakers,  with  95 
percent  confidence  bounds  for  this  data  population,  resulted  in  the  estimate  of 
overall  relationship  between  intelligibility  and  bit  error  rate  presented  in  Figure  8. 
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RANDOM  BIT  ERROR  RATE  (PERCENT) 


Figure  8.  Linear  Hegression  Model  Expressing  Variation  in  Intelligibility  With 
Bit  Error  Rate,  V  ith  95%  Confidence  Limits  for  Individual  Speaker's  Intelligibility 
Scores,  for  the  DUD  LPC-10  Vocoder  in  the  Composite  E4B  Acoustic  Environment 


The  composite  data  indicated  that  an  average  intelligibility  score  of  80.2  (fair)  can 
be  expected  with  no  bit  errors,  and  that  bit  errors  will  cause  intelligibility  to  fall 
4-1/2  points  for  each  percent  bit  errors.  The  regression  line  indicates  that  aver¬ 
age  intelligibility  will  become  unacceptable  (will  drop  below  a  score  of  70)  at  bit 
error  rates  greater  than  2.3  percent. 

6.2  lnt«'lliuiliilil\  of  ltitlmiliial  Speaker-  in  |  li<>  Couipn-ile  Kill  \e<m-lie  Km  ironmeiil 

Statistical  tests  of  regression  models  fitted  to  intelligibility  scores  obtained 
for  individual  speakers  in  the  composite  E4B  acoustic  environment  indicated  that 
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slopes  of  individual  speaker's  regression  lines  for  vocoder  intelligibility  in  this 
environment  did  not  differ  significantly.  However,  on  the  basis  of  a  common  slope, 
differences  in  the  elevations  (origins)  of  individual  speaker's  regression  lines  were 
significant.  A  new  calculation  based  on  a  common  slope  led  to  the  result  indicated 
in  Figure  9.  The  result  illustrates  how  different  individuals  varied  in  intelligibi- 
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Figure  9.  Regression  Lines  Expressing  Variation  in  Intelligibility  With  Bit  Error 
Rate  of  Individual  Speakers,  in  Tests  of  the  DOD  LPC-10  Vocoder  in  the  E4B  Com¬ 
posite  Acoustic  Environment 


lity  under  this  combination  of  vocoder,  acoustic  environment,  and  bit  errors.  Ori¬ 
gins  of  regression  lines  for  individual  speakers  varied  from  84.4  for  the  "best" 
speaker  to  77.2  for  the  "poorest,  "  with  a  common  slope  of  -4.  51. 
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7.  VOICE  QUALITY  RATINGS  (DAM/CAE  SCORES)  OBTAINED  FOR  THE  DOD 
LPC-10  VOCODER  IN  THE  E4B  COMPOSITE  ACOUSTIC  ENVIRONMENT 


Overall  quality  ratings  (DAM/CAE  scores)  from  testing  the  DOD  narrowband 
LPC-10  vocoder  in  the  acoustic  environments  of  the  E4B  Battlestaff  Compartment, 
NCA  Compartment,  and  Briefing  Room  are  listed  in  Tables  A4,  A5,  and  A6.  The 
overall  quality  score  is  a  combination  of  two  component  scores,  a  score  for  signal 
quality  (DAM/CSA)  and  a  score  for  background  quality  (DAM/CBA).  (See  Section 
3,3.)  Tables  of  these  component  scores  obtained  for  each  of  the  acoustic  environ¬ 
ments  are  presented  in  Tables  A7  through  A 12.  The  scores  for  overall  quality 
were  analyzed,  and  linear  regression  models  for  the  relationship  between  quality 
scores  and  bit  error  rate  were  calculated.  The  quality  scores  obtained  for  indivi¬ 
dual  speakers  are  shown  as  scatter  diagrams  in  relation  to  the  regression  lines  and 
the  95  percent  confidence  bounds  for  the  data.  For  data  for  the  Battlestaff  Com¬ 
partment,  see  Figures  10  and  11;  for  the  NCA  Compartment,  see  Figures  12  and  13; 
and  for  the  Briefing  Room  data,  see  Figure  14.  A  comparison  of  the  regression 
lines  reveals  the  similarity  of  the  three  acoustic  environments  in  their  effects  on 
vocoder  quality.  This  similarity  is  illustrated  in  Figure  15. 
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Figure  10.  Male  Speaker's  Speech  Quality  (DAM/CAE)  Scores  vs 
Bit  Error  Rate,  in  Relation  to  Linear  Regression  Modeling,  for 
the  DOD  LPC-10  Vocoder  in  the  E4B  Battlestaff  Compartment 
Acoustic  Environment 


E4B 

Bait  1  •staff  CoapartMnt 

ENVIRONMENT 


EXPECTATION. 

□AM  <CAE)  SCORE 
FEM.  SPEAKERS 


**  41.  7-2.  16  x  BERX 


RANDOM  BIT  ERROR  RATE  (PERCENT) 

Figure  11.  Female  Speaker's  Speech  Quality  (DAM /CAE)  Scores 
vs  Bit  Error  Rate,  in  Relation  to  Linear  Regression  Modeling,  for 
the  DOD  LPC-10  Vocoder  in  the  E4B  Battlestaff  Compartment 
Acoustic  Environment 
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Figure  12.  Male  Speaker's  Speech  Quality  (DAM/CAE)  Scores  vs 
Bit  Error  Rate,  in  Relation  to  Linear  Regression  Modeling,  for 
the  DOD  LPC-10  Vocoder  in  the  E4B  NCA  Compartment  Acoustic 
Environment 
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Figure  15.  Comparison  of  Expected  Variation  in  Speech  Quality 
(DAM/CAE  Scores)  With  Bit  Error  Rate  for  the  DOD  LPC-10 
Vocoder  Tested  in  the  Acoustic  Environments  of  the  Three  E4B 
Compartments 


7.1  Composite  Kr-ult:  Voice  Quality  vs  BU  Error  Rate 

Overall  voice  quality  scores  obtained  for  the  vocoder  in  the  three  acoustic  en¬ 
vironments  were  pooled,  and  a  regression  model  was  calculated  for  the  composite 
data.  The  resulting  linear  regression  line  and  95  percent  confidence  bounds  for  the 
data  population  are  shown  in  Figure  16.  With  no  bit  errors,  average  quality  score 
for  the  DOD  LPC-10  vocoder  in  the  composite  acoustic  environment  is  expected  to 
be  43.  3  (fair).  Under  bit  error  conditions,  the  average  score  is  expected  to  fall 
2.3  points  for  each  percent  bit  error  rate.  Thus,  at  a  1 -percent  error  rate,  an 
average  quality  score  of  41.0  (poor)  is  estimated. 

7.2  Composite  Result:  Individual  Speaker's  Voice  Quality  Scores 

Statistical  tests  comparing  regression  models  fitted  to  voice  quality  scores  vs 
error  rate  for  individual  speakers  indicated  that  slopes  of  the  regression  lines  cal¬ 
culated  for  individual  speakers  were  not  significantly  different,  but  the  elevations 
(origins)  of  regression  lines  for  individual  speaker's  scores  did  differ  by  significant 
amounts.  A  new  regression  model  and  origins  of  the  regression  lines  representing 
individual  speaker's  scores  were  calculated  based  on  a  common  slope.  This  result 
is  presented  in  Figure  17.  The  origins,  estimates  of  the  voice  quality  scores  of 
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Figure  16.  Linear  Regression  Model  Expressing  Variation  in  Speech  Quality 
(DAM/CAE  Scores)  With  Bit  Error  Rate,  With  95%  Confidence  Limits  for  Indivi¬ 
dual  Speaker's  Voice  Quality  Scores,  for  the  DOD  LPC-10  Vocoder  in  the  Compo¬ 
site  E4B  Acoustic  Environment 


individual  speakers  with  no  bit  errors,  varied  from  45.4  for  the  best  speaker  to 
41.3  for  the  poorest,  with  a  common  slope  of  -2.3. 

8.  CATEGORIES  OF  INTELLIGIBILITY  ANI)  VOICE  yi  \I.ITY  PERFORMANCE  FOR 
THE  DOD  LPC-10  VOCODER  IN  TIIE  E  IB  COMPOSITE  AGO!  STIC 
ENVIRONMENT 

The  expected  variation  in  intelligibility  and  voice  quality  of  the  DOD  LPC-10 
vocoder  in  the  E4B  acoustic  environment  is  summarized  in  Table  5,  which  presents 
numerical  values  and  categories  for  expected  intelligibility  performance  (as  shown 
in  Figure  9)  and  voice  quality  performance  (as  shown  in  Figure  16).  The  statisti¬ 
cal  models  fitted  to  the  performance  data  indicate  that,  with  no  bit  errors,  an  av¬ 
erage  intelligibility  score  of  80.  2  can  be  expected,  and  that  95  percent  of  indivi¬ 
dual  speaker's  intelligibility  scores  will  be  distributed  between  87.7  (good)  and 
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Table  5.  Predicted  Scores  and  C  ategories  of  Performance  for  the  i  >!  *  I  pc  -10 
Vocoder  in  the  E4B  (C  omposite)  Acoustic  Environment,  V\  ith  the  Resident  Micro¬ 
phone  With  95%  Confidence  Bounds  for  Scores  From  a  Population  of  Male  and  Fe¬ 
male  Speakers 
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9.  COMPARATIVE  PERFORMANCE:  THE  DOD  EPC-IO  NARROWBAND  VOCODER 
l  NDER  QI  IET  CONDITIONS 

It  is  instructive  to  compare  the  measured  performance  of  the  DOD  standard 
LPC-10  vocoder  under  "ideal"  conditions,  that  is,  with  speakers  using  a  high- 
quality  dynamic  microphone  in  a  quiet  environment,  with  the  measurements  of  per¬ 
formance  obtained  in  the  E4B  acoustic  environment.  DRT  intelligibility  scores 
from  earlier  tests  of  this  vocoder  under  quiet  conditions  are  listed  in  Table  A 13, 
and  voice  quality  ratings  (DAM/CAE  scores)  are  listed  in  Table  A14.  Separate 
components  of  the  overall  quality  score,  the  scores  for  signal  quality  (DAM/CSA), 
and  the  scores  for  background  quality  (DAM/CBA)  are  listed  in  Tables  A15  and  A16. 

9.1  Vocoder  Inlrlligihilitv  I  ndrr  Ideal  Condition* 

A  linear  regression  model  calculated  for  the  relationship  between  intelligibi¬ 
lity  and  bit  error  rate  from  pooled  intelligibility  scores  for  male  and  female 


speakers  resulted  in  the  regression  line  and  95  percent  confidence  bounds  for  the 
data  shown  in  Figure  18.  An  average  intelligibility  score  of  86.8  (moderate)  is 
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Figure  18.  Linear  Regression  Model  Expressing  Variation  in  Intelligibility  With 
Bit  Error  Rate,  With  95%  Confidence  Limits  for  Individual  Speaker's  Intelligibility 
Scores,  for  the  DOD  LPC-10  Vocoder  in  a  Quiet  Environment 


expected  with  no  bit  errors,  with  a  regression  line  having  a  slope  of  -2.  1.  Statis¬ 
tical  tests  comparing  the  regression  models  calculated  for  individual  speaker's 
scores  in  relation  to  bit  error  rate  indicated  that  slopes  of  the  individual  regres¬ 
sion  functions  did  not  differ  significantly.  However,  the  origins  of  the  lines  for 
individual  speaker's  scores  differed  by  significant  amounts.  A  new  calculation 
based  on  a  common  slope  of  -2. 13  established  values  for  the  origins  of  the  regres¬ 
sion  lines  for  the  six  male  and  six  female  speakers'  scores.  The  origins,  repre¬ 
senting  the  expected  intelligibility  with  no  bit  errors,  varied  from  91.6  for  the 
best  speaker  to  82.2  for  the  poorest.  The  average  intelligibility  of  male  speakers 
was  almost  5  points  higher  than  that  of  female  speakers.  This  is  illustrated  in 
Figure  19. 
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Figure  19.  Regression  Lines  Expressing  Variation  in  Intelligibility  With  Bit  Error 
Rate  of  Individual  Speakers,  in  Tests  of  the  DOD  LPC-10  Vocoder  in  a  Quiet  Envi¬ 
ronment 


9.2  Vocoder  Voice  Quality  Ratings  (DAM /CAE  Scores)  I’nder  Ideal  Conditions 


For  the  pooled  data  representing  quality  of  the  DOD  LPC-10  vocoder  tested 
with  male  and  female  speakers  under  quiet  conditions,  a  linear  regression  model 
was  calculated  for  the  relationship  between  voice  quality  scores  and  bit  error  rate. 
The  regression  line  and  95  percent  confidence  bounds  for  the  data  are  shown  in 
Figure  20.  With  no  bit  errors,  the  expected  average  quality  score  is  46,0  (fair), 
with  a  regression  line  slope  of  -2.  6.  The  regression  lines  expressing  variation  of 
quality  scores  with  bit  error  rate  of  individual  speakers  were  found  to  have  a  com¬ 
mon  slope  but  significantly  different  origins,  and  a  new  regression  model  based  on 
this  finding  established  the  variation  of  quality  with  bit  error  rate  for  individual 
speakers.  These  regression  lines  are  shown  in  Figure  21.  The  origins,  estimating 
voice  quality  scores  with  no  bit  errors,  varied  from  47.  7  for  the  best  speaker  to 
44.9  for  the  poorest,  with  a  common  slope  of  -2.62.  The  average  quality  scores 
for  male  speakers  were  2. 1  points  higher  than  those  for  female  speakers. 


37 


QUIET 

ENVIRONMENT 


0  1  2  3  4  5  6 


RANDOM  BIT  ERROR  RATE  (PERCENT) 


Figure  20.  Linear  Regression  Model  Expressing  Variation  in 
Speech  Quality  (DAM/CAE  Scores),  With  95%  Confidence  Limits 
for  Individual  Speaker's  Voice  Quality  Scores,  for  the  DOD  LPC- 
10  Vocoder  in  a  Quiet  Environment 
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Figure  21.  Regression  Lines  Expressing  Variation  in  Speech 
Quality  (DAM /CAE  Scores)  With  Bit  Error  Rate,  for  Individual 
Speakers,  in  Tests  of  the  DOD  LPC-10  Vocoder  in  a  Quiet 
Environment 


will  be  34.2  (very  poor),  with  95  percent  of  the  scores  of  indiv'dual  speakers  re¬ 
presented  by  this  sample  distributed  between  38.2  (poor)  and  30.2  (very  poor). 


10.  COMPARATIVE  PERFORMANCE:  WIDEBAND  CONTENT  OI  S  VARIABLE- 
SLOPE  DELTA  MODI  LATION  (CVSD)  VOICE  PROCESSORS 

Previous  studies  have  established  intelligibility  scores  and  voice  quality  rat¬ 
ings  of  wideband  CVSD  processors  operating  at  16  K  bps  and  32  K  bps  data  rates. 
Those  studies  were  for  ideal  conditions,  that  is,  speakers  were  in  a  quiet  environ¬ 
ment. 

10. 1  I  nlcfligiiiiiity  of  tin*  CVSD- 16  Processor  in  a  Quiet  Kmirorinient 

DRT  intelligibility  scores  obtained  from  tests  of  a  16K  bps  CVSD  processor  at 
0,  1/2,  1,  2,  and  5  percent  bit  error  rates  are  listed  in  Table  A 16.  These  data 
were  used  in  calculating  a  linear  regression  model  leading  to  the  regression  line 
and  95  percent  confidence  bounds  for  the  data  shown  in  Figure  22.  With  no  bit  er¬ 
rors,  the  average  intelligibility  is  expected  to  be  91.4  (very  good).  The  calculated 
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Figure  22.  Linear  Regression  Model  Expressing  Variation  in 
Intelligibility  With  Bit  Error  Rate,  With  95%  Confidence  Limits 
for  Individual  Speaker's  Intelligibility  Scores,  for  the  Wideband 
CVSD-16  Processor  in  Quiet  Environment 


slope  of  the  regression  function  is  -1.  1.  Regression  functions  modeling  individual 
speaker's  intelligibility  vs  bit  error  rate  were  found  to  have  a  common  slope,  but 
differ  significantly  in  the  origins  of  the  lines.  A  regression  model  calculated  on 
the  basis  of  a  common  slope  determined  regression  functions  for  individual  speak¬ 
ers,  illustrated  in  Figure  23.  The  origins,  representing  the  expected  intelligibi¬ 
lity  score  with  no  bit  errors,  ranged  from  94.  9  for  the  best  speaker  to  89.  0  for  the 
poorest.  Male  speakers'  intelligibility  scores  tended  to  exceed  those  of  female 
speakers. 


RANDOM  BIT  ERROR  RATE  (PERCENT) 


Figure  23.  Regression  Lines  Expressing  Variation  in  Intelligibi¬ 
lity  With  Bit  Error  Rate  of  Individual  Speakers,  in  Tests  of  the 
Wideband  CVSD-16  Processor  in  Quiet  Environment 


10.2  Voire  Qnulit>  Raling>  (DAM/LAE  Scores)  for  the  CVSD-16  Processor  in  a 

tjiiicl  Km  ironment 

Voice  quality  scores  from  three  independent  tests  of  the  CVSD-16  processor 
are  listed  in  Table  A 17.  Corresponding  scores  for  the  components  of  the  overall 
quality  score,  the  scores  for  signal  quality  (DAM/CSA),  and  the  scores  for  back¬ 
ground  quality  (CAM/CBA)  are  listed  in  Tables  A18  and  A19.  The  overall  quality 
(DAM/CAE)  scores  for  male  and  female  speakers  were  pooled,  and  a  regression 
model  was  calculated  for  the  combined  data,  leading  to  the  result  illustrated  in 
Figure  24,  The  figure  presents  the  regression  line  and  95  percent  confidence 
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Figure  24.  Linear  Regression  Model  Expressing  Variation  in 
Speech  Quality  (DAM/CAE  Scores)  With  Bit  Error  Rate,  With 
95%  Confidence  Limits  for  Individual  Speaker's  Voice  Quality 
Scores,  for  the  Wideband  CVSD-16  Processor  in  a  Quiet  Envi¬ 
ronment 


bounds  for  this  collection  of  scores.  The  statistical  model  indicated  that,  with  no 
bit  errors,  an  average  quality  score  of  51.  1  (moderate)  was  obtained,  with  a  slope 
of  -1.  9  for  the  regression  line.  Regression  lines  calculated  for  scores  vs  bit  er¬ 
ror  rate  of  individual  speakers  did  not  differ  significantly  in  the  slopes,  but  signi¬ 
ficant  differences  were  indicated  for  the  origins.  A  new  regression  model  was 
calculated  on  the  basis  of  a  common  slope.  The  resulting  regression  functions  for 
individual  speaker's  voice  quality  scores  vs  error  rate  are  shown  in  Figure  25. 
Origins  of  the  lines,  estimating  individual  speaker's  voice  quality  scores  with  no 
bit  errors,  varied  from  53.  6  for  the  best  speaker  to  49.4  for  the  poorest,  with  a 
common  slope  of  -1.  94.  Quality  ratings  for  male  speakers  tended  to  exceed  those 
obtained  for  female  speakers. 


10.3  Categories  of  liilflligilMlit)  and  Voire  Oualit\  Performance  for  the  CYSD-lft 
Processor 

The  statistical  models  fitted  to  the  intelligibility  data  and  voice  quality  data  for 
the  CVSD-16  processor,  illustrated  graphically  in  Figures  22  and  24,  provided  the 
numerical  values  presented  in  Table  7,  The  table  gives  the  descriptive  categories 
associated  with  the  levels  of  performance.  With  no  bit  errors,  it  is  estimated  that 
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Figure  25.  Regression  Lines  Expressing  Variation  in  Speech 
Quality  (DAM/CAE  Scores)  With  Bit  Error  Rate  of  Individual 
Speakers,  in  Tests  of  the  Wideband  CVSD-16  Processor  in  a 
Quiet  Environment 


95  percent  of  the  intelligibility  scores  of  individual  speakers  from  a  population  re¬ 
presented  by  this  sample  will  be  between  96.  9  (excellent)  and  86.  0  (moderate).  At 
a  3  percent  error  rate,  the  expected  average  intelligibility  score  is  88.2  (good), 
with  an  estimate  that  95  percent  of  individual  speaker's  intelligibility  scores  will 
fall  between  93.  6  (very  good)  and  82.  7  (fair).  Voice  quality  performance  fell  into 
categories  somewhat  below  those  for  intelligibility.  With  no  bit  errors,  it  is  esti¬ 
mated  that  95  percent  of  the  quality  scores  of  individual  speakers  will  fall  between 
57.2  (good)  and  44.9  (fair).  At  a  3  percent  error  rate,  the  expected  average  qual¬ 
ity  score  is  45.2  (fair),  and  the  95  percent  confidence  bounds  for  individual  speak¬ 
er's  quality  scores  extend  from  51.4  (moderate)  to  39.  1  (poor). 

10.  I  lnl«‘lligil)ilit>  of  the  F\SI)-32  in  a  i^uirl  Kn\ iroiimrnl 

Intelligibility  scores  obtained  from  tests  of  the  32  K  bps  CVSD  processor  under 
bit  error  conditions  in  a  quiet  acoustic  environment  are  listed  in  Table  A20.  A 
linear  regression  model  calculated  from  these  scores  established  the  relationship 
between  performance  and  bit  error  rate;  the  regression  line  and  95  percent  confi¬ 
dence  bounds  for  the  data  are  shown  in  Figure  26.  Without  bit  errors,  an  average 
intelligibility  score  of  93.6  (very  good)  is  expected,  with  a  regression  slope  of 


Table  7.  Predicted  Scores  and  Categories  of  Performance  for  the  CVSD-16  Pro¬ 
cessor  in  a  Quiet  Acoustic  Environment  (Dynamic  Microphone)  With  95%  Confidence 
Bounds  for  Scores  From  a  Population  of  Male  and  Female  Speakers 


-0.  75;  that  is,  scores  are  expected  to  drop  by  three-fourths  of  a  point  with  each 
percent  of  bit  errors.  As  with  the  other  processors,  the  regression  functions  cal¬ 
culated  for  individual  speaker's  intelligibility  scores  vs  bit  error  rate  were  found 
to  have  a  common  slope,  but  differ  significantly  in  regard  to  the  elevations  of  the 
regression  lines.  A  new  regression  model  was  calculated  on  the  basis  of  this  find¬ 
ing.  The  result,  shown  in  Figure  27,  gave  values  of  the  origins  of  the  regression 
lines  that  varied  from  96.  6  for  the  best  speaker  to  90.  7  for  the  poorest. 

10.5  Voice  Quality  Rating*  (DAM/C. \E  Score*)  for  the  (ASI142  Processor  in  a 
Quiet  Environment 

Two  independent  tests  of  the  voice  quality  of  the  CVSD-32  processor  with  input 
speech  from  a  quiet  acoustic  environment  resulted  in  the  voice  quality  scores  listed 
in  Table  A21.  The  regression  model  fitted  to  this  data  indicated  the  relationship 
between  quality  scores  and  bit  error  rate  that  is  shown  in  Figure  28.  With  this 
group  of  speakers,  an  average  quality  score  of  58.  3  (very  good)  is  expected  with  no 
bit  errors,  with  a  slope  of  the  regression  function  of  -2.4.  Comparisons  of  re- 
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Figure  26.  Linear  Regression  Model  Expressing  Variation  in  In¬ 
telligibility  With  Bit  Error  Rate,  With  95%  Confidence  Limits  for 
Individual  Speaker's  Intelligibility  Scores,  for  the  Wideband 
CVSD-32  Processor  in  a  Quiet  Environment 
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Figure  27.  Regression  Lines  Expressing  Variation  in  Intelligibi¬ 
lity  With  Bit  Error  Rate  for  Individual  Speaker's  Intelligibility 
Scores  for  the  Wideband  CVSD-32  Processor  in  a  Quiet  Environment 
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Figure  28.  Linear  Regression  Model  Expressing  Variation  in 
Speech  Quality  (DAM/CAE  Scores)  With  Bit  Error  Rate,  With  95% 
Confidence  Limits  for  Individual  Speaker's  Voice  Quality  Scores, 
From  Tests  of  the  Wideband  CVSD-32  Processor  in  a  Quiet  Envi¬ 
ronment 


gression  models  calculated  from  individual  speaker's  scores  indicated  that,  while 
the  elevations  of  the  regression  lines  based  on  individual  speaker's  scores  differed 
significantly,  the  differences  in  slopes  of  the  lines  were  not  significant.  A  new 
regression  model  was  calculated  based  on  this  finding,  and  the  result  is  shown  in 
Figure  29.  Origins  of  regression  lines  for  individual  speakers,  estimating  the 
voice  quality  score  with  no  bit  errors,  varied  from  61.7  for  the  best  speaker  to 
54.0  for  the  poorest,  with  a  common  slope  of  -2.43. 

10.6  Categories*  of  Intelligibility  and  Voire  Quality  Performance  for  the  CV  SD-32 
Processor  in  a  Quiet  Environment 

The  statistical  models  for  the  variation  in  performance  with  bit  error  rate  il¬ 
lustrated  in  Figures  26  and  28  provided  the  numerical  values  and  associated  de¬ 
scriptive  categories  shown  in  Table  8.  With  no  bit  errors,  it  is  estimated  that 
95  percent  of  the  intelligibility  scores  of  individual  speakers  will  fall  between  98.  7 
(excellent)  and  88.  5  (good),  and  voice  quality  scores  will  fall  between  66. 1  (excel¬ 
lent)  and  50.  5  (moderate).  At  a  3  percent  error  rate,  it  is  predicted  that  intelligi¬ 
bility  scores  will  have  95  percent  probability  of  falling  between  96.4  (excellent) 
and  86.3  (moderate),  while  quality  scores  will  fall  between  58.  8  (very  good)  and 
43.3  (fair). 
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Figure  29.  Regression  Lines  Expressing  Variation  in  Speech 
Quality  (DAM /CAE  Scores)  vs  Bit  Error  Rate  for  Individual 
Speakers  in  Tests  of  the  Wideband  CVSD-32  Processor  in  Quiet 
Environment 


I  I.  COMPARISONS  OF  VOICE  PROCESSORS  AND  TEST  CONDITIONS 


Comparisons  of  performance  capabilities  of  the  voice  processors  are  facili¬ 
tated  with  the  linear  regression  models  for  performance  vs  bit  error  rate.  In  Fig¬ 
ure  30,  predicted  intelligibility  performance  is  compared  with  bit  error  rate  for 
the  DOD  narrowband  LPC-10  vocoder  in  the  E4B  acoustic  environment  and  in  a 
quiet  environment.  Results  from  tests  of  an  LPC-10  vocoder  lacking  bit  error  de¬ 
tection  and  correction  are  also  shown  for  comparison,  1  and  for  the  functions  esti¬ 
mating  performance  of  the  CVSD-16  and  CVSD-32  processors.  Comparisons  of 
voice  quality  performance  with  bit  error  raxe  are  presented  in  Figure  31.  Relative 
ranking  of  the  processors  is  the  same  according  to  intelligibility  and  voice  quality. 
With  increasing  bit  error  rate,  however,  the  differences  in  quality  of  the  LPC-10 
vocoder  in  the  quiet  and  in  the  E4B  environment  tend  to  become  smaller,  while  the 
differences  in  intelligibility  performance  tend  to  become  larger.  The  linear  re¬ 
gression  coefficients  are  compared  in  Table  9,  which  also  lists  the  coefficients 
calculated  for  the  best  and  the  poorest  speakers  for  each  processor. 


Table  8.  Predicted  Scores  and  Categories  of  Performance  for  the  CVSD-32  Pro¬ 
cessor  in  a  Quiet  Acoustic  Environment  (Dynamic  Microphone)  With  95%  Confidence 
Bounds  for  Scores  From  a  Population  of  Male  and  Female  Speakers 


1  1.1  Diagnostic  Intelligibility  Scores  vs  Bit  Error  Rate  for  the  I)OD  LPC-10  Vocoder 

The  variations  in  scores  obtained  for  the  separate  intelligibility  features  of  the 
DRT  provide  insights  regarding  the  relative  vulnerability  of  the  different  phonetic 
features  to  bit  error  effects  and  noise  effects.  For  this  reason,  separate  linear 
regression  models  were  calculated  for  each  of  the  intelligibility  features  tested  by 
the  DRT.  The  results  from  the  performance  data  on  the  vocoder  in  the  E4B  com¬ 
posite  acoustic  environment  are  summarized  in  Figure  32;  the  corresponding  re¬ 
sults  for  vocoder  performance  in  a  quiet  environment  are  summarized  in  Figure  33. 
For  both  conditions,  the  feature  "sibilation"  obtained  the  highest  scores,  and  the 
features  "graveness"  and  "sustention"  received  the  poorest  scores,  but  were  under 
much  greater  impairment  in  the  noise  environment.  Details  of  the  variations  in 
scores  for  sustention  are  shown  in  Figure  34,  and,  for  graveness,  in  Figure  35. 

Research  on  improving  the  intelligibility  of  these  phonetic  features  could  pro¬ 
vide  the  most  payoff  in  overall  improvement  in  intelligibility  performance  of  the  vo- 
vocoder  under  these  impairments.  The  regression  coefficients  determined  for  in¬ 
dividual  features  are  compared  in  Table  10. 


Table  9.  Comparison  of  Voice  Processors:  Average  Performance  vs  Bit  Error 
Rate  for  a  Population  of  Male  and  Female  Speakers 


r  - — - - - 1 

Processor,  Environment, 

Intelligibility  Estimate 

Voice  Quality  Estimate 

and  Microphone 

(Avg  DRT  Score) 

(Avg  DAM/CAE  Score) 

DOD  LPC-10  Vocoder 

S(DRT>  *  80.  2-4.51xBER(%) 

S1DAM/CAE)  =  43.  3-2.  30xBER(%) 

in  E4B  (Composite) 

Best: 

«  84.4-4.51xBER<%) 

=  45.4-2.30xBER<%) 

Environment,  with 

Poorest: 

-  77.2-4.  51xBER(%) 

=  41.3-2.  30xBEH(%) 

resident  microphone 

DOD  LPC-10  Vocoder 

Average  soeaker: 

S(DRT)  .  86.  8-2. 13xBER<%) 

SIDAM/CAE)  =  46.  0-2.  82xl)ER{%> 

in  Quiet,  with 

Best: 

-  91.6-2.  !3xBER(%) 

-  47.7-2.  62xliKR(%) 

dynamic  microphone 

Poorest: 

=  82.2-2.  13xBER<%) 

=  44.9-2.  62xftfcft(%) 

CVSD-16  Processor 

Average  speaker: 

S(DRT)  .  91,4-1. 09xBER(%) 

S(DAM/CAE)  =  51.  l-1.84xREii(%) 

in  Quiet,  with 

Best: 

-  94.  9-1.  09xBER(%) 

-  53.  6  - 1 . 94xD  £}{(%) 

dynamic  microphone 

Poorest: 

-  88.  0-1. 08xBEH<%) 

=  49.4-1. 94xDKR<%) 

CVSD-32  Processor 

Average  speaker: 

S(DRT)  =  93.  6-0.75xBER<%) 

S(DAM/CAE)  =  58.  3-2.43xHt:l(<%) 

in  Quiet,  with 

Best: 

=  96.  6-0.  75xBER(%) 

*  61.7-2.  43xBER(%) 

dynamic  microphone 

Poorest: 

=  90.7-0.75xBER(%) 

*  54.0-2.43xHER<%) 

1  1.2  Signal  (Jualilv  (DAM /(ISA)  and  Background  Ouulitv  (DAM/GIlA)  Scores  u. 

Bit  Error  Rate 

The  variations  in  signal  quality  (DAM/CSA)  scores  with  bit  error  rate  are 
compared  for  the  LPC-10  vocoder  in  the  E4B  acoustic  environment  and  the  vocod¬ 
er  in  quiet  in  Figure  36.  The  variation  in  background  quality  (DAM/CBA)  scores 
with  bit  error  rate  are  compared  in  Figure  37.  Signal  quality  and  background 
quality  were  both  lowered  in  the  E4B  noise  environment.  The  comparison  of  sig¬ 
nal  quality  scores  indicated  higher  scores  for  male  speakers  in  the  quiet,  but  this 
ranking  was  reversed  in  the  E4B  acoustic  noise  environment,  where  the  female 
speakers  tended  to  receive  the  higher  scores  for  signal  quality.  The  scores  for 
background  quality  were  virtually  identical  for  male  and  female  speakers. 

12.  DISCI  SSION  OF  FINDINGS 

The  acoustic  noise  environment  of  the  E4B  caused  a  significant  lowering  of 
speech  intelligibility  scores  and  voice  quality  scores  obtained  for  the  DOD  LPC-10 
vocoder.  Among  the  factors  that  have  been  found  to  affect  intelligibility  scores 
and  voice  quality  scores  are  the  particular  microphones  and  the  particular  combi¬ 
nations  of  speakers  used  in  conducting  tests. 
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Figure  30.  Comparison  of  Regression  Lines  Estimating  Intelligi¬ 
bility  vs  Bit  Error  Rates  for  the  DOD  LPC-10  Vocoder  in  the  E4B 
and  Quiet  Acoustic  Environments  With  the  CVSD-16  and  CVSD-32 
Processors 
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Figure  31.  Comparison  of  Regression  Lines  Estimating  Voice 
Quality  vs  Bit  Error  Rate  for  the  DOD  LPC-10  Vocoder  in  the 
E4B  and  Quiet  Acoustic  Environments,  and  the  CVSD-16  and 
CVSD-32  Wideband  Processors 
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Figure  32.  Comparison  of  Regression  Lines  Estimating  Scores  for 
the  Intelligibility  Features  vs  Bit  Error  Rate  for  the  DOD  LPC-10 
Vocoder  in  the  E4B  Composite  Acoustic  Environment 
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Figure  33.  Comparison  of  Regression  Lines  Estimating  Scores  for 
the  Intelligibility  Features  vs  Bit  Error  Rate  for  the  DOD  LPC-10 
Vocoder  in  a  Quiet  Acoustic  Environment 
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Figure  34.  Comparison  of  Regression  Lines  Estimating  Scores  for 
the  Sustention  Intelligibility  Feature  vs  Bit  Error  Rate  for  the  DOD 
L PC-10  Vocoder  in  Quiet  and  E4B  Environments 
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Figure  35.  Comparison  of  Regression  Lines  Estimating  Scores  for 
the  Graveness  Intelligibility  Feature  vs  Bit  Error  Rate  for  the  DOD 
LPC-10  Vocoder  in  Quiet  and  E4B  Environments 
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12.1  Ti'sl  Micmpli.m. 


It  has  been  observed  that  a  noise  cancelling  microphone  tends  to  produce 
slightly  poorer  intelligibility  scores  and  voice  quality  scores  in  tests  in  a  quiet  en¬ 
vironment  than  a  high-quality  dynamic  microphone.  However,  this  advantage 
quickly  disappears  when  testing  in  all  but  the  most  benign  noise  environments  be¬ 
cause  of  the  improvement  in  signal -to -noise  ratio  obtained  with  the  noise  cancelling 
microphone.  The  high  quality  microphone  has  been  found  to  be  the  best  choice  for 
the  ideal  quiet  conditions,  but,  for  noise  environments  like  those  found  on  the  E4B, 
it  is  likely  that  the  Roanwell  noise  cancelling  microphone  used  for  these  tests  is  at 
or  very  near  to  the  optimum  design.  *  This  was  not  true  of  the  microphone  on  the 
EC-135,  as  discussed  in  Section  12.7. 


12.2  Tol  Speaker*. 


Digital  voice  processors,  especially  narrowband  processors,  have  been  found 
to  be  very  speaker-variable.  Only  very  limited  comparisons  can  be  made  of  pro¬ 
cessors  or  test  conditions  unless  speakers  have  been  standardized.  For  tests  of 
the  effects  of  noise  environments  on  vocoders  and  wideband  voice  processors,  the 
DOD  Digital  Voice  Processor  Consortium  has  settled  on  a  "standard"  speaker  set 
of  three  male  and  three  female  speakers.  The  Consortium  speaker-set  was  used 
for  these  tests,  expanded  by  adding  three  more  male  standard  speakers  to  reduce 
experimental  errors.  Thus,  the  nine  speakers  used  for  these  tests  were  a  control 
set  in  the  sense  that  the  data  on  speech  intelligibility  and  voice  quality  scores  ob¬ 
tained  with  them  is  extensive  enough  to  permit  comparisons  and  statistical  tests. 

There  is  no  assurance,  however,  that  these  speakers  are  a  sample  typical  of 
command  and  control  communicators  for  factors  such  as  regional  accents,  age, 
voice  pitch,  etc.  That  uncertainty  is  mitigated  by  the  fact  that  command  and  con¬ 
trol  communicators  tend  to  have  extensive  experience  in  communicating  under  ad¬ 
verse  conditions,  able  to  adjust  their  voices  to  cope  with  noisy  surroundings  when 
speaking  and  also  able  to  understand  speech  under  adverse  conditions  when  listen¬ 
ing.  Therefore,  it  is  likely  that  "naive"  speakers  and  listeners  from  a  general  ci¬ 
vilian  population  would  obtain  results  like  those  from  these  studies,  while  experi¬ 
enced  military  communicators  would  probably  exceed  these  predictions.  Further 
research  is  required  to  resolve  this  question. 


*  An  experimental  second-order  noise  cancelling  microphone  developed  for  over¬ 
coming  severe  acoustic  noise  environments  was  found  to  give  poorer  results 
than  those  presented  here. 
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12.3  Random  Bit  Errors  vs  Block  Errors 

Considerable  evidence  exists  to  indicate  that  voice  processor  tests  with  ran¬ 
dom  bit  errors  represent  a  "worst  case,  "  and  that  block  errors,  or  clustered  er¬ 
rors,  that  usually  occur  in  transmitting  digital  data  over  wireline  and  radio  chan¬ 
nels  have  a  smaller  effect  on  intelligibility  and  voice  quality  than  randomly  distri¬ 
buted  errors.  From  this  point  of  view,  the  predicted  effects  of  bit  errors  are  con¬ 
servative  in  the  sense  that  data  errors  occurring  in  clusters,  rather  than  in  random 
distribution,  will  tend  to  degrade  intelligibility  and  voice  quality  less  than  these 
forecasts. 

12.  1  Categories  of  Intelligibility  and  Voice  Quality  Performance 

Categorizations  of  intelligibility  and  quality  scores  as  good,  poor,  unaccept¬ 
able,  etc. ,  represent  a  consensus  reached  by  the  DOD  Digital  Voice  Processor 
Consortium  from  contacts  with  a  number  of  workers  in  this  field.  These  categori¬ 
zations  have  not  yet  been  validated  by  testing  with  military  communicators.  Sched¬ 
uled  research  studies  using  conversational  testing  are  expected  to  clarify  the  as¬ 
signments  of  categories. 

1 2.o  Results  Not  Applicable  to  I  .PC- 10  Vocoders  That  Deviate  From  the  DOD 
Standard  Algorithm 

The  comparison  illustrated  in  Figure  30  of  Vocoder  intelligibility  with  and  with¬ 
out  bit  error  correction  under  bit  error  conditions  illustrates  the  fact  that  vocoders 
deviating  from  the  DOD  standard  algorithm  will  probably  obtain  speech  intelligibility 
and  voice  quality  scores  different  from  those  presented  here.  The  lack  of  the  bit¬ 
error  detection  and  correction  features  could  significantly  reduce  performance  in 
comparison  with  these  values. 

12.6  Comparison  of  Results  From  Live  Recordings  With  Results  From  Simulations 

Intelligibility  scores  obtained  for  the  DOD  LPC-10  vocoder  with  speech  record¬ 
ings  prepared  live  aboard  the  aircraft  during  flight  are  compared  with  scores  ob¬ 
tained  from  tests  based  on  the  simulation  of  the  acoustic  environment  of  the  E4B 
Battlestaff  Compartment  in  Figure  38.  The  cluster  of  scores  and  95  percent  confi¬ 
dence  bounds  for  the  set  of  data  obtained  with  the  simulation  with  speakers  using  the 
resident  microphone  are  compared  with  the  scores  obtained  with  the  live  recordings 
when  several  microphones  were  used  aboard  the  aircraft.  None  of  the  microphones 
gave  any  clear  advantage  over  the  Roanwell  model  used  aboard  the  aircraft.  Fur¬ 
ther  details  of  the  various  microphones  have  been  reported  by  Teacher.  ®  The  dis¬ 
tribution  of  scores  illustrates  that  live  intelligibility  tests  prepared  in  the  field  tend 
to  exhibit  greater  variability  than  do  test  recordings  prepared  under  closely  con- 
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E4B  BattlestaEf  Compartment 


Figure  38.  Comparison  of  Intelligibility  Scores  and  95%  Con¬ 
fidence  Bounds  for  Individual  Speaker's  Scores  for  the  DOD 
LPC-10  Vocoder  in  the  Simulated  E4B  Battlestaff  Compart¬ 
ment  Acoustic  Environment  With  Scores  From  Live  Record¬ 
ings  Aboard  the  Aircraft 

trolled  conditions.  Even  with  these  fluctuations,  there  is  general  agreement  be¬ 
tween  results  from  the  live  recordings  aboard  the  aircraft  and  the  simulations  of 
the  acoustic  environment. 

12.7  Comparison  of  Id  t It  and  Acoustic  Environment?. 

Vocoder  intelligibility  performance  in  the  acoustic  environment  of  the  EC-135 
was  similar  to  that  obtained  for  the  E4B,  probably  because  of  the  similarity  in  the 
background  noise  level  and  power  spectrum.  However,  the  voice  quality  (DAM/- 
CAE)  scores  obtained  for  the  vocoder  in  the  EC-135  acoustic  environment  were 
lower  than  scores  obtained  for  the  E4B.  This  is  illustrated  in  Figure  39.  I.rwer 
quality  ratings  were  probably  related  to  the  differences  in  microphones;  thespeech- 
to-noise  ratio  provided  by  the  pressure  microphone  used  on  the  EC-135  was 
poorer  than  that  ratio  provided  by  the  noise  cancelling  microphone  used  on  the  E4E. 

18.  CONCI.I  SIONS  \M>  RECOMMENDATIONS 

The  E4B  Battlestaff  Compartment,  NCA  Compartment,  and  Briefing  Room 
acoustic  environments  during  flight  were  found  to  be  not  significantly  different  it 
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RANDOM  BIT  ERROR  RATE  (PERCENT) 


Figure  39.  Comparison  of  Voice  Quality  (DAM /CAE)  Scores  vs 
Bit  Error  Rate  for  the  DOD  LPC-10  Vocoder  in  the  EC-135 
Acoustic  Environment  and  in  the  E4B  Composite  Acoustic  Envi¬ 
ronment 


their  effects  on  speaker's  speech  intelligibility  and  voice  quality  tested  with  the 
DOD  standard  LPC-10  narrowband  vocoder. 

Intelligibility  and  voice  quality  of  the  vocoder  were  significantly  poorer  with 
speakers  in  the  acoustic  environment  of  the  E4B  aircraft  than  with  speakers  in  a 
quiet  environment. 

Tests  with  male  and  female  speakers  in  the  E4B  acoustic  environment  indicated 
that  average  intelligibility  of  the  vocoder  operating  with  no  data  errors  was  80.  2 
(fair),  with  95  percent  confidence  bounds  for  individual  speaker's  intelligibility 
scores  extending  from  87.7  (good)  to  72.7  (very  poor).  At  1  percent  random  bit- 
errors,  average  intelligibility  was  75.7  (poor),  with  95  percent  confidence  bounds 
for  individual  speaker's  intelligibility  extending  from  83.2  (moderate)  to  68.2  (un¬ 
acceptable). 

Average  voice  quality  rating  for  the  vocoder  with  speakers  in  the  E4B  acoustic 
environment  was  43.3  (fair)  in  the  absence  of  data  errors,  with  95  percent  confi¬ 
dence  bounds  for  individual  speaker's  quality  ratings  extending  from  47.2  (fair)  to 
39.3  (poor).  At  a  2  percent  error  rate,  the  tests  indicated  an  average  voice  quality 
rating  of  38.  7  (poor),  with  95  percent  confidence  bounds  for  individual  speaker's 
quality  ratings  extending  from  42.6  (fair)  to  34.  7  (very  poor). 
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At  error  rates  of  1  x  10  ,  the  intelligibility  and  quality  of  the  vocoder  would 

be  essentially  the  intelligibility  and  quality  quoted  for  the  error-free  condition. 

Evaluation  data  indicated  that,  at  error  rates  of  0.  6  percent  or  less,  95  per¬ 
cent  of  the  individual  speakers  from  a  population  for  which  these  were  a  repre¬ 
sentative  sample  would  obtain  intelligibility  and  quality  above  the  unacceptable 
category. 

Vocoders  deviating  from  the  DOD  standard  algorithm  may  not  reach  these 
performance  levels,  especially  if  they  lack  the  bit-error  detection  and  correction 
features. 

A  variation  on  LPC-10  vocoder  design  known  as  "piecewise-LPC'1  has  design 
features  that  make  it  intrinsically  less  vulnerable  to  bit  error  effects  and  acous¬ 
tic  noise  effects  than  a  conventional  LPC-10  vocoder.  ^  Our  recommenda¬ 
tions  are  that  further  research  be  done  to  complete  the  research  in  optimizing  the 
PLPC  vocoder  design  and  incorporating  bit  error  correction  features,  and  that 
comparative  evaluations  be  conducted  to  determine  its  effectiveness  in  alleviating 
the  degradation  of  performance  caused  by  noise  and  bit  error  effects. 

13.  Roberts,  J.E.,  and  Wiggins,  R.H.  (1976)  Piecewise  linear  predictive  coding 
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Appendix  A 

Intelligibility  and  Voice  Quality  Scores 


78.0  75.4 


(83.  5) 


Table  Bl.  List  of  Master  Tapes:  E4B  Battlestaff  Compartment  Acoustic 
Environment  (Roanwell  240100001  N/C  Microphone) 


K10-1.  1-A  ♦ 

RH  323A 

K10-1.  1-B* 

CT 

301B 

K10-1.  1-C* 

vw 

335A 

JE  314B 

AS 

306B 

KS 

315B 

CH  309B 

BV 

303A 

MP 

331A 

►— * 

o 

»-* 

to 

i* 

RH  323B 

K10-1.2-B 

CT 

301A 

K10-1.2-C 

VW 

335B 

JE  315A 

AS 

305A 

KS 

316B 

K10-1.3-A 

RH  324A 

K10-1.  3-B 

CT 

302A 

K 10-1. 2-C 

VW 

336A 

JE  315B 

AS 

306A 

KS 

316A 

CH  3 10A 

BV 

303B 

MP 

332B 

K10-1.4-A 

RH  324B 

K10-1.4-B 

CT 

302B 

K10-1.4-C 

VW 

336B 

JE  314A 

AS 

305B 

KS 

315A 

CH  310B 

BV 

304A 

MP 

331B 

♦This  tape  includes  the  Three-Speaker  DAM  following  the  last  DRT. 


Table  B2.  List  of  Master  Tapes:  E4B  NCA  Compartment  Acoustic 
Environment  (Roanwell  Model  240100001  Noise -Cancelling  Microphone) 


K10-2.  1-A* 

RH  32  5A 

K 10-2.  IB* 

CT 

308B 

K10-2.  1-C* 

VW  301A 

JE  312A 

AS 

312A 

KS  317B 

CH  308A 

BV 

309A 

MP  333A 

K10-2.2-A 

RH  326A 

K10-2.2-B 

CT 

307B 

K10-2.2-C 

VW  3  02 A 

JE  312B 

AS 

311A 

KS  318B 

CH  308B 

BV 

309B 

MP  334A 

K10-2.3-A 

RH  326B 

K10-2.3-B 

CT 

308A 

K10-2.3-C 

VW  302B 

JE  313B 

AS 

312B 

KS  318A 

CH  307B 

BV 

310A 

MP  334B 

K10-2.4-A 

RN  325B 

K10-2.4-B 

CT 

307A 

K10-2.4-C 

VW  301B 

JE  3 13A 

AS 

31  IB 

KS  317A 

CH  307A 

BV 

31 0B 

MP  333B 

♦This  tape  includes  the  Three-Speaker  DAM  following  the  last  DRT. 
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i) 

K10-3. 1-A* 

RH  327B 

JE  317A 

CH  305A 

K10-3. 1-B* 

CT  313A 

AS  329B 

BV  315A 

K10-3.2-A 

RH  327B 

JE  318A 

CH  305B 

K 10-3.  2B 

CT  313B 

AS  330B 

BV  315B 

K10-3.3-A 

RH  328A 

JE  318B 

CH  306B 

K10-3.3-B 

CT  314A 

AS  330A 

BV  316A 

K10-3.4-A 

RH  327A 

JE  3 17B 

CH  306A 

K10-3.4-B 

CT  314B 

AS  329A 

BV  316B 

♦This  tape  includes  the  Three-Speaker  DAM  following 
the  last  DRT. 


Table  B4.  List  of  Master  Tapes:  EC-135  Battlestaff  Compartment  Acoustic 
Environment  (Roanwell  Model  60150  Microphone) 


Kll-1. 1-A* 

RH  330A 

JE  309B 

CH  304A 

Kll-1. 1-B* 

CT  317A 

AS  321B 

BV  323A 

Kll-1.  1-C* 

VW  3  03 A 

KS  31 IB 

MP  320A 

K11-1.2-A 

RH  329B 

JE  309A 

CH  303B 

Kll-1.  2-B 

CT  3 17B 

AS  322E 

BV  324A 

Kll-1.  2-C 

VW  303B 

KS  312B 

MP  319A 

Kll-1. 3-A 

RH  329A 

JE  310A 

CH  304B 

Kll-1. 3-B 

CT  318A 

AS  322A 

BV  324B 

Kll-1. 3-C 

VW  304B 

KS  311A 

MP  320B 

Kll-1. 4-A 

RH  330B 

JE  310B 

CH  3  03 A 

Kll-1. 4-B 

CT  318B 

AS  32 1A 

BV  323B 

Kll-1. 4-C 

VW  304A 

KS  312A 

MP  319B 

♦This  tape  includes  the  Three -Speaker  DAM  following  the  last  DRT. 

.‘-W.V.V. VA 


Kll-2. 1-A* 

RH  331A 

Kll-2.  1  -B* 

CT  319A 

Kll-2.  1-C* 

VW  305A 

JE  307B 

AS  328B 

KS  3 14A 

CH  301B 

BV  326A 

MP  32 IB 

K11-2.2-A 

RH  331B 

Kll-2. 2-B 

CT  32 OB 

Kll-2. 2-C 

VW  306B 

JE  308B 

AS  327B 

KS  313B 

CH  302B 

BV  325B 

MP  322B 

Kll-2.  3 -A 

RH  332A 

Kll-2. 3-B 

CT  320A 

Kll-2.  3-C 

VW  306A 

JE  308A 

AS  327A 

KS  3 13A 

CH  301A 

BV  326B 

MP  32 1A 

Kll-2. 4-A 

RH  332B 

Kll-2.  4-B 

CT  3 19B 

Kll-2. 4-C 

VW  305B 

JE  307A 

AS  328A 

KS  3 14B 

CH  3  02 A 

BV  32 5A 

MP  322A 

*This  tape  includes  the  Three-Speaker  DAM  following  the  last  DRT 
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